Papers with sequence-level contrastive learning
Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune pre-trained models to downstream tasks are limited by labeled examples. |
| Approach: | They propose to apply post-training on unlabeled task data before fine-tuning by contrastive learning that considers either token-level or sequence-level similarity. |
| Outcome: | Empirical results show that contrastive masked language modeling surpasses other methods in few-shot settings without the need for data augmentation. |
GECSum: Generative Evaluation-Driven Sequence Level Contrastive Learning for Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization is a technique in natural language processing that involves generating a summary of a source document by creating new sentences and phrases. |
| Approach: | They propose a sequence-level contrastive learning framework that leverages the semantic understanding capabilities of the abstractive model itself to evaluate summary in reference-based settings. |
| Outcome: | The proposed framework outperforms the state-of-the-art in four summarization datasets. |